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The findings from this review do not refiect the fuii body of 
research evidence on the Teach for America program. 


What is this study about? 

The study examined whether students taught by 
teachers in the Teach for America (TFA) and The 
New Teacher Project Teaching Feiiows (Teaching 
Feiiows) programs had greater mathematics 
achievement than students taught by teachers who 
were not in either of these programs. This WWC 
report focuses on the TFA intervention, and a 
separate single study review provides information 
about the Teaching Feiiows intervention. 

The TFA study included two cohorts of students in 
grades 6-1 2 (one from the 2009-1 0 school year and 
one from the 201 0-1 1 school year). Students came 
from eight states, 1 1 school districts, and 45 schools, 

Within each school, students were randomly 
assigned to a mathematics classroom taught by 
either a TFA teacher or a comparison teacher who 
did not enter teaching through a highly selective 
alternative route. In total, the TFA study randomly 
assigned over 6,000 students to either a TFA 
teacher or a comparison teacher. 

Mathematics achievement was assessed for 
students in grades 6-8 using scores from state- 
required assessments. For students in grades 
9-12, the authors administered end-of-course math 
assessments developed by the Northwest Evalua- 
tion Association (NWEA). 


Features of Teach for America (TFA) 


TFA is a highly selective alternative certification 
program for new teachers. Each program recruits, 
selects, trains, places, and provides support to 
teachers in low-income, high-need schools across 
the country. 

TFA’s primary focus for recruitment is recent college 
graduates from prestigious colleges and universities. 
Recruits receive training during a 5-week summer 
institute. Program directors provide support to their 
teachers during the school year. TFA teachers are 
required to commit to teaching for 2 years. 


WWC Rating 


The research described in this 
report meets WWC group design 
standards without reservations 

The TFA study is a well-executed randomized 
controlled trial with low levels of sample attrition."* 


What did the study find? 

The study authors found, and the WWC confirmed, 
that TFA teachers were more effective than compari 
son teachers in raising the mathematics achieve- 
ment of their students (effect size of 0.07).® 
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Appendix A: Study details 

Clark, M. A., Chiang, H. S., Silva, T., McConnell, S., Sonnenfeld, K., Erbe, A., & Puma, M. (2013). 

The effectiveness of secondary math teachers from Teach For America and the Teaching 
Feiiows programs (NCEE 2013-4015). Washington, DC: National Center for Education Evaluation 
and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. 


Setting 

The Teach For America (TFA) study was conducted in eight states, 1 1 school districts, and 


45 schools. 

Study sample 

In each participating schooi, researchers matched two or more math classrooms so that at 
least one classroom would be taught by a TFA teacher, and at least one classroom would be 
taught by a teacher who did not participate in a highly selective alternative route to certifica- 
tion (such as TFA or Teaching Feiiows). Students in grades 6-12 were randomly assigned 
to these classrooms, typically at the beginning of the school year. However, students who 
entered a participating school after the beginning of the school year were randomly assigned 
to classrooms through rolling random assignment. The TFA study was conducted with two 
cohorts of students in the 2009-10 and 2010-1 1 school years. The study authors noted that 
the TFA study sample was comprised primarily of middle school classrooms (75%). 

The TFA study was conducted in 45 schools located in eight states, with 1 1 1 classroom 
matches covering 66 intervention and 70 comparison teachers. For this study, 3,075 students 
were randomly assigned to a TFA teacher, and 3,103 students were randomly assigned to a 
comparison teacher. The analytic sample included 2,292 students in the intervention group 
and 2,281 students in the comparison group. 

Intervention 

group 

The intervention group consisted of classrooms of TFA teachers. TFA recruits, selects, trains, 
places, and provides support to teachers in low-income, high-need schools across the coun- 
try. Participants must commit to teaching in the program for 2 years. TFA’s primary focus for 
recruitment is recent college graduates from prestigious colleges and universities. The program 
is highly selective {TFA accepts about 12% of applicants) and has a substantial application and 
selection process that includes a written application, a telephone interview, an analysis and 
writing exercise, an in-person interview, a monitored group discussion, and a 5-minute sample 
teaching lesson. Once selected, TFA provides training through a 5-week summer institute that 
precedes the start of the first year of teaching and three other shorter training activities. TFA 
recruits also attend regional induction programs for several days, during which they learn more 
about the TFA program, as well as the region and school they will be teaching in. The TFA 
study’s intervention group included teachers in the process of completing their 2-year teaching 
commitments, as well as those who continued teaching after their TFA commitment ended. 

Comparison 

group 

Classrooms were eligible to be in the comparison group if their teachers were certified through 
a route other than TFA, Teaching Feiiows, or a similar program. 
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Outcomes and 
measurement 


Support for 
implementation 


Reason for 
review 


For students in grades 6-8, the authors measured mathematics achievement using scores 
obtained on state- required assessments. For students in grades 9-12, the authors admin- 
istered end-of-course mathematics assessments developed by the Northwest Evaluation 
Association (NWEA). All test scores were converted to z-scores to facilitate comparisons 
across classrooms, schools, and districts. For a more detailed description of these outcome 
measures, see Appendix B. 

TFA recruits are required to (a) complete 30-40 hours of independent study, (b) attend a 
5-week pre-service summer institute that includes curriculum Instruction and 40 hours of 
practice teaching, and (c) participate in post-institute training activities (Round Zero) that 
include 25 hours of online coursework. Most TFA teachers are also required to enroll In a 
state’s alternative certification program. TFA program directors provide support to the 
teachers during the school year by leading classroom observations with feedback and 
one-on-one and group meetings. 

This study was identified for review by the WWC by receiving media attention. 
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Appendix B: Outcome measure for the mathematics achievement domain 


Mathematics achievement 


Mathematics achievement 


Mathematics assessments 

For students in grades 6-8, the authors measured mathematics achievement using scores from state-required 
assessments. These assessments were administered at the end of the schooi year in which the students were 
randomiy assigned. 

For students in grades 9-12, the authors administered end-of-course mathematics assessments deveioped 
by the Northwest Evaiuation Association (NWEA). These computer-adaptive assessments covered generai high 
schooi math, Aigebra i. Geometry, or Aigebra ii, depending upon the course content. 

Aii test scores were converted to z-scores. For state assessments administered to middie schooi students, the 
reference popuiation was the fuii popuiation of students in the same state, year, and grade who took the same 
assessment. For NWEA end-of-course assessments administered to high schooi students, the reference 
popuiation was the NWEA’s nationwide norming sampie for that assessment. 
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Appendix C: Study findings for the mathematics achievement domain, TFA study 





Mean 

(standard deviation) 

WWC calculations 


Domain and 
outcome measure 

Study 

sample 

Sample 

size 

Intervention 

group 

Comparison 

group 

Mean 

difference 

Effect 

size 

Improvement 

index 

p-value 

Mathematics achievement 

Mathematics assessments 

Cohorts 1 
and 2 

45 schools/ 
136 teachers/ 
4,573 students 

nr 

nr 

nr 

0.07 

-f3 

0.00 

Domain average for mathematics achievement 




0.07 

+3 

Statistically 

significant 


Table Notes: For mean difference, effect size, and improvement index vaiues reported in the tabie, a positive number favors the intervention group and a negative number favors 
the comparison group. The effect size is a standardized measure of the effect of an intervention on student outcomes, representing the average change expected for aii students 
who are given the intervention (measured in standard deviations of the outcome measure). The improvement index is an aiternate presentation of the effect size, refiecting the 
change in an average student’s percentiie rank that can be expected if the student is given the intervention. The statisticai significance of the study's domain average was 
determined by the WWC. nr = not reported. Note: the study reports adjusted means, which are reflected in the effect size estimates, but it does not report the raw means. 

Study Notes: No corrections for clustering or multiple comparisons and no difference-in-differences adjustment were needed. The p-value presented here was reported in 
the original study. The effect size is the impact estimate from the study because the outcome was scaled to be in standard deviation units. This study is characterized as having a 
statistically significant positive effect because the effect for at least one measure within the domain is positive and statistically significant, and no effects are negative and statisti- 
cally significant. For more information, please refer to the WWC Standards and Procedures Flandbook, version 3.0, 
pages 26-27. 
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Endnotes 

■' Single study reviews examine evidence published in a study (supplemented, if necessary, by information obtained directly from the 
author[s]) to assess whether the study design meets WWC evidence standards. The review reports the WWC’s assessment of whether 
the study meets WWC evidence standards and summarizes the study findings following WWC conventions for reporting evidence on 
effectiveness. This study was reviewed using the single study review protocol, version 2.0. A quick review of this study was released 
on November 7, 2013, and this report is the follow-up review that replaces that initial assessment. 

^ Absence of conflict of interest: This study was conducted by staff from Mathematica Policy Research. Therefore, Mathematica 
reviewers were not involved in the WWC review of this study. 

® The study also examined the impact of The New Teachers Project Teaching Fellows (Teaching Fellows) relative to a separate com- 
parison group that was formed by random assignment. The findings from that analysis are reported in a separate single study review 
by the WWC because the impacts of the two programs were measured separately. The authors also noted that the study was not 
designed to compare the effectiveness of Teach For America relative to Teaching Fellows. 

The WWC verified low study attrition through an author query. 

® The study also examined impacts by subgroups including school level (e.g., middle and high) and by type of comparison teacher 
(e.g., traditionally certified and alternatively certified). However, the study did not provide enough information to determine a rating for 
these analyses, so they are not presented in this single study review. 

Recommended Citation 

U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2014, May). 

WWC review of the report: The effectiveness of secondary math teachers from Teach for America and the 
Teaching Fellows programs. Retrieved from http://whatworks.ed.gov 
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Glossary of Terms 

Attrition 


Clustering adjustment 
Confounding factor 

Design 
Domain 
Effect size 

Eligibility 

Equivalence 

Improvement index 


Multiple comparison 
adjustment 

Quasi-experimental 
design (QED) 

Randomized controlled 
trial (RCT) 

Single-case design 
(SCD) 

Standard deviation 


Statistical significance 
Substantively important 


Attrition occurs when an outcome variable is not avaiiabie for aii participants initiaiiy assigned 
to the intervention and comparison groups. The WWC considers the total attrition rate and 
the difference in attrition rates across groups within a study. 

If intervention assignment is made at a cluster level and the analysis is conducted at the student 
level, the WWC will adjust the statistical significance to account for this mismatch, if necessary. 

A confounding factor is a component of a study that is completely aligned with one of the 
study conditions, making it impossible to separate how much of the observed effect was 
due to the intervention and how much was due to the factor. 

The design of a study is the method by which intervention and comparison groups were assigned. 
A domain is a group of closely related outcomes. 

The effect size is a measure of the magnitude of an effect. The WWC uses a standardized 
measure to facilitate comparisons across studies and outcomes. 

A study is eligible for review if it falls within the scope of the review protocol and uses either 
an experimental or matched comparison group design. 

A demonstration that the analysis sample groups are similar on observed characteristics 
defined in the review area protocol. 

Along a percentile distribution of students, the improvement index represents the gain 
or loss of the average student due to the intervention. As the average student starts at 
the 50th percentile, the measure ranges from -50 to +50. 

When a study includes multiple outcomes or comparison groups, the WWC will adjust 
the statistical significance to account for the multiple comparisons, if necessary. 

A quasi-experimental design (QED) is a research design in which subjects are assigned 
to intervention and comparison groups through a process that is not random. 

A randomized controlled trial (RCT) is an experiment in which investigators randomly assign 
eligible participants into intervention and comparison groups. 

A research approach in which an outcome variable is measured repeatedly within and 
across different conditions that are defined by the presence or absence of an intervention. 

The standard deviation of a measure shows how much variation exists across observations 
in the sample. A low standard deviation indicates that the observations in the sample tend 
to be very close to the mean; a high standard deviation indicates that the observations in 
the sample are spread out over a large range of values. 

Statistical significance is the probability that the difference between groups is a result of 
chance rather than a real difference between the groups. The WWC labels a finding statistically 
significant if the likelihood that the difference is due to chance is less than 5% (p < 0.05). 

A substantively important finding is one that has an effect size of 0.25 or greater, regardless 
of statistical significance. 


Please see the WWC Procedures and Standards Handbook (version 3.0) for additional details. 
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